Concurrent Analysis of Physiologic Variables for the Assessment of the Affective State of a Computer User

Much progress has been made during the last 40 years in the quest to improve the interaction of humans with computers. While new modalities of communication between computers and their users continue to be found and enhanced (e.g., speech recognition for human-to-computer communication and speech synthesis for computer-tohuman communication), the nature of the exchange between computers and users remains, for the most part, dry and mechanistic. The emerging field of Affective Computing seeks to advance Human-Computer Interaction (HCI) by enabling computers to interact with users in ways appropriate to their affective states. However, a major prerequisite to the fulfillment of the promise of Affective Computing is the development of efficient mechanisms for Affective Sensing, i.e., the ability of the computer to assess the affective state of its user, particularly when it shifts to uncomfortable states, such as stress. Our research pursues the use of three physiological signals: Blood Volume Pulse (BVP), Galvanic Skin Response (GSR) and Pupil Diameter (PD), to automatically monitor the level of stress in computer users. This paper reports on the hardware and software instrumentation development and signal processing approach used to detect the stress level of a subject interacting with a computer, within the framework of a specific experimental task, which is called the ‘Stroop Test’. The Stroop Effect is evoked by the mismatch between the font color and the meaning of a certain word (name of a color), displayed to the experimental subject. For this experiment, a computer game was implemented and adapted to make the subject experience this effect, while his/her BVP, GSR and PD signals were continuously recorded. Three data processing techniques were applied to extract effective attributes of the stress level of the subjects throughout the experiment. Current results indicate that there exists a correspondence between changes in those three signals and the shift in the emotional states when stress stimuli are applied to the interaction environment.